Utilizing an abstraction relation document in grey-box testing approach

This paper describes the test oracle generation from an abstraction relation document that is documented using Parnas’s Module Documentation (MD) method. This work is part of on-going research that addresses the problem of improving the effectiveness of fault detection. We focus our work on unit/module testing where each module may consist of several programs. The aim of our project is to investigate the strategies and techniques to automate module testing. In particular, we investigate the use of MD that is written in standard mathematical notation to automate the process of test oracle generation and test execution

Module documentation based testing using grey-box approach

Testing plays an important role to assure the quality of software. Testing is a process of detecting errors that can be highly effective if performed rigorously. The use of formal specifications provides significant opportunity to develop effective testing techniques. Grey-box testing approach usually based on knowledge obtains from specification and source code while seldom the design specification is concerned. In this paper, we propose an approach for testing a module with internal memory from its formal specification based on grey-box approach. We use formal specifications that are documented using Parnas’s Module Documentation (MD) method. The MD provides us with the information of external and internal view of a module that can be useful in grey-box testing approach

Comparison of software complexity metrics in measuring the complexity of event sequences

One of the main challenges in software development is the complex structure of a system. The software development for event sequences is complex. It is a challenge to define a complexity metric for event sequences application. Lack of knowledge in complexity metric can lead to issues such as rises in software cost and delays in project timing. Numerous complexity metrics have been proposed and published, such as information flow complexity, lines of code, function points, and unique complexity metric. However, in the context of the event sequences, most of the research focuses on measuring web graphs, measuring the web traffic and how the complexity of the web impacts the customer. In this paper, the researchers studied and compared five different software complexity metrics. This paper describes the on-going research that addresses the issue to produce a unique weight to prioritise event sequences test cases

A preliminary investigation towards test suite optimization approach for enhanced state-sensitivity partitioning

Testing is crucial in software development. Continuous researches being done to discover effective approaches in testing that capable to detect faults despite of reducing cost. Previous work in State-Sensitivity Partitioning (SSP) technique, which based on all-transition coverage criterion, has been introduced to avoid exhaustively testing the entire data states of a module by partitioning it based on state’s sensitivity towards events, conditions and action. The test data for that particular module testing is in form of event sequences (or test sequence) and sets of test sequences in test cases will perform SSP test suite. The problem occurs in SSP test suite is data state redundancy that leads towards suite growth. This paper aims to discuss an initial step of our ongoing research in enhancing prior SSP test suite. Our work will try to find out the best way in removing redundant data state in order to minimize the suite size but yet capable to detect faults introduced by five selective mutation operators effectively as the original suite

Test case prioritization technique for event sequence test cases based on redundancy factor

Software testing is often used to verify and validate the output of the system to confirm that no discrepancy has taken place throughout the development phase. Test case prioritization (TCP) is one of the techniques applied to modify the order of test cases based on best test scenarios and to prioritize them. The main objectives of the TCP are to increase the effectiveness of the testing process, while reducing time and cost, which would increase when the system reaches a certain level of complexity. Numerous TCP techniques have been proposed in the past; however, only a handful of researches were truly focused on TCP techniques for test cases involving the sequence of events. TCP technique for sequence of events is more complex compared to the conventional code-based application due to the properties of the sequence of events. The size of the sequence of events’ test cases can be infinite and large sized test cases have considerable degrees of redundancy. This means that there is a possibility for these test cases to have combinations of events with a large input parameter. Redundancy is one of the major issues that have been discussed by previous researchers. This paper proposes a technique that can detect the redundancy within the test suites and produce a unique weight value. This paper will also present how test cases were prioritized based on the obtained unique weight value. The experiment results obtained indicates that the prioritized test suite is effective compared with the original test suite. The effectiveness of the proposed approach is evaluated using Average Percentage of Faults Detected (APFD)

Factor determination in prioritizing test cases for event sequences: A systematic literature review

The generation of test cases is a challenging phase in software testing. The process of test case generation becomes more expensive and time-consuming when the test suites become larger. Many researchers have proposed the test case prioritization (TCP) technique to schedule test cases, so that those with the highest priority are executed first before lower priority test cases. One of the performance goals of TCP is the rate of fault detection, which is a measure of how quickly faults are detected within the testing phase. However, the existing TCP technique has some limitations. This paper presents the results of a systematic literature review (SLR) of relevant primary studies as evidence of the existence of TCP in the area of event sequences. Consequently, five major techniques and 10 factors were identified and analysed. This study aims to review and identify techniques and factors that influence the process of assigning weight values in TCP processes. The proposed factors need to be evaluated in terms of their contribution to the performance of the TCP technique. Some researchers believe that a combination of factors might be required to produce unique weights during the TCP processes. Nevertheless, most studies applied the random method or did not provide any information regarding the same weight value issues

An experimental assessment of module documentation-based testing

Context: Testing a module that has memory using the black-box approach has been found to be expensive and relatively ineffective. Instead, testing without knowledge of the specifications (white-box approach) may not be effective in showing whether a program has been properly implemented as stated in its specifications. We propose instead a grey-box approach called Module Documentation-based Testing or MD-Test, the heart of which is an automatic generation of the test oracle from the external and internal views of the module. Objective: This paper presents an empirical analysis and comparison of MD-Test against three existing testing tools. Method: The experiment was conducted using a mutation-testing approach, in two phases that assess the capability of MD-Test in general and its capability of evaluating test results in particular. Results: The results of the general assessment indicate that MD-Test is more effective than the other three tools under comparison, where it is able to detect all faults. The second phase of the experiment, which is significant to this study, compares the capabilities of MD-Test and JUnit-black using the test evaluation results. Likewise, an analysis of the test evaluation results shows that MD-Test is more effective and efficient, where MD-Test is able to detect at least the same number of faults as, or is at par with, the black-box approach. Conclusion: It is concluded that test evaluation using grey-box approach is more effective and efficient that the black-box approach when testing a module that has memory

Systematic literature review on search based software testing

The use of random search is very poor at finding solutions when those solutions occupy a very small part of the overall search space. Test data may be found faster and more reliably if the search is given some guidance. This work is a paper that explains the application of metaheuristic techniques in search-based software testing. The paper systematically review 47 papers selected randomly from online databases and conference proceeding based on the metaheuristic search techniques that have been most widely applied to problem solving, the different fitness function used for test data selection in each of the metaheuristic technique, and the limitation in the use of each search-based technique for software testing. It was found that GA outperformed its counterparts SA, HC, GP and random search approaches in generating test data automatically, different approaches were used to make sure that test data are selected within shorter period of time and also with wider coverage of the paths based on the fitness function, and most of the limitations of the articles are the handling of complex data types, like array, object types, and branch coverage. The paper also provides areas of possible future work on the use of metaheuristic techniques in search-based software testing

The effectiveness of an enhanced weighted method with a unique priority value for test case prioritization in regression testing

Regression testing is an important and costly strategy in software testing. To decrease its cost, many methods have been proposed including the test case prioritization methods. The aim of the prioritization methods is to define an ideal order of test cases that allows for higher coverage and early fault detection with minimal number of executed test cases. However, the problem with most of the existing test case prioritization methods is the random sorting of test cases when two or more test cases record equal priority values. In this paper, the effectiveness of an enhanced weighted method using a unique priority value, UniVal, is proposed. Unival prioritizes test cases based on code coverage criteria with information from history of previous executions. In addition, a controlled experiment was executed, and the results were statistically analyzed to assess the effectiveness of the proposed method. The results indicates better performance with regard to prioritize test cases and achieve higher APFD values